Nonstationary value iteration in controlled Markov chains with risk-sensitive average criterion
نویسندگان
چکیده
منابع مشابه
Empirical Bayes Estimation in Nonstationary Markov chains
Estimation procedures for nonstationary Markov chains appear to be relatively sparse. This work introduces empirical Bayes estimators for the transition probability matrix of a finite nonstationary Markov chain. The data are assumed to be of a panel study type in which each data set consists of a sequence of observations on N>=2 independent and identically dis...
متن کاملRecent Results in Controlled Markov Chains with Risk Sensitive Average Criteria: the Vanishing Discount Approach
Countable state space Markov cost/ reward chains, satisfying a Lyapunov-t ype stability condition, are considered in this work. For an infinite planning horizon, risk sensitive (exponential) discounted and average cost criteria are considered. The main contribution is the development of a vanishing discount approach to relate the discounted criterion problem with the average criterion one, as t...
متن کاملControlled Markov chains with risk-sensitive criteria: Average cost, optimality equations, and optimal solutions
We study controlled Markov chains with denumerable state space and bounded costs per stage. A (long-run) risk-sensitive average cost criterion, associated to an exponential utility function with a constant risk sensitivity coe1⁄2cient, is used as a performance measure. The main assumption on the probabilistic structure of the model is that the transition law satis®es a simultaneous Doeblin cond...
متن کاملValue Iteration in a Class of Average Controlled Markov Chains with Unbounded Costs: Necessary and Sufficient Conditions for Pointwise Convergence
This work concerns controlled Markov chains with denumerable state space, (possibly) unbounded cost function, and an expected average cost criterion. Under a Lyapunov function condition, together with mild continuity-compactness assumptions, a simple necessary and sufficient criterion is given so that the relative value functions and differential costs produced by the value iteration scheme con...
متن کاملA Characterization of the Optimal Risk-sensitive Average Cost in Finite Controlled Markov Chains
This work concerns controlled Markov chains with finite state and action spaces. The transition law satisfies the simultaneous Doeblin condition, and the performance of a control policy is measured by the (long-run) risk-sensitive average cost criterion associated to a positive, but otherwise arbitrary, risk sensitivity coefficient. Within this context, the optimal risk-sensitive average cost i...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Journal of Applied Probability
سال: 2005
ISSN: 0021-9002,1475-6072
DOI: 10.1017/s0021900200000991